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Report Summary 


The Better Math Teaching Network (BMTN) is a networked improvement community of researchers, 
teachers, and instructional leaders from New England who use improvement science principles to 
increase the number of students who are actively and deeply engaged in algebra content. The BMTN’s 
second year of implementation occurred during the 2017-18 school year. Key findings include: 


¢ Continued high teacher participation and engagement. As with the first cohort, BMTN 
teachers actively participated in roughly 100 hours of individual, small group, and whole 
group network activities. 


e Increased opportunities for deep student engagement. As reported both by teachers and 
students, BMTN classrooms provided increasing opportunities for students to deepen their 
understanding of algebra over the course of the 2017-18 year. 


e Progress made towards achieving the network aim. The BMTN made steady progress towards 
meeting its aim of increasing the number of algebra students who connect, justify, and solve 
with depth. 


* Continued deepening of student-centered instruction. Through math content study groups, 
the selection of instructional tasks, and the testing of instructional routines, BMTN classrooms 
had a stronger focus on developing mathematical relationships. 


e Experienced BMTN teachers key to spreading network learning. Through their interactions 
with new BMTN members and the sharing of refined instructional routines, returning BMTN 
teachers helped accelerate the learning of new BMTN teachers and share the work of BMTN 
to teachers outside the network. 


Better Math Teaching Network: Lessons Learned from Year2 | 3 


The Better Math Teaching Network (BMTN) is a networked improvement community (NIC) of researchers, 
teachers, and instructional leaders who share a common aim: to increase the number of students in New 


England who deeply engage in algebra content. The heart of our network is high school algebra teachers, 
who actively test, refine, and share instructional routines that are aligned with our network aim. These 
teachers, who represent every state in New England, primarily work in rural or urban schools and work 
with economically disadvantaged students. A research team from the American Institutes for Research 
(AIR) comprises the hub and leads the network, which consisted of 51 teachers and 10 instructional 
leaders during the 2017-18 school year and is the focus of this report.' 


Our work is guided by principles of improvement science, an applied scientific approach designed to 
help organizations solve complex problems and improve performance through iterative, rapid-cycle 
testing. Though used in recent decades in industries outside of education, this approach is relatively 
new to education.? Unlike interventions that focus on achieving high levels of fidelity and consistency 
of implementation, improvement science focuses on understanding variation in implementation. 
Improvement scientists expect variation in implementation and view it as an opportunity to target 
ongoing improvement efforts and further strengthen implementation. Thus, BMTN teachers approach 
their work with the understanding that even well-conceived, well-implemented instructional routines 
will fail at certain points in the lesson. Those failures provide opportunities to further strengthen the 
routine the next time it is implemented. 


Our NIC is structured so that teachers and researchers collaborate frequently—both in person and 
virtually—to share what is being learned through the testing and refinement of instructional routines, 
and to identify resources and supports for ongoing improvement. BMTN teachers participated in 
about 100 hours of organized in-person and virtual meetings during the 2017-18 school year and 
many reported spending more time informally collaborating with other network members outside 

of network meetings. 


Building on broader research in mathematics education,’ including a study carried out previously by the 
members of the network hub,° our network has identified three domains of deep student engagement 
that comprise our aim. Though not mutually exclusive, BMTN teachers focus their improvement efforts 
on one of these three domains: connect, justify, and solve. 


* Connect. Making connections among mathematical algorithms, concepts, and application 
to real-world contexts, where appropriate. 


° Justify. Communicating and justifying mathematical thinking as well as critiquing the 
reasoning of others. 


¢ Solve. Making sense of and solving challenging math problems that extend beyond rote 
application of algorithms. 


Promoting deep student engagement in these three areas runs counter to standard practices in U.S. 
math classrooms, which tend to focus more heavily on developing students’ procedural skill through 
teacher-directed approaches. We aim to use student-centered approaches to develop students’ ability to 
make connections, justify their thinking, and solve complex problems.® We focus on these three domains 
within algebra, considered a STEM “gatekeeper” course, to support larger efforts to improve U.S. student 
achievement in mathematics’ and increase the number of U.S. high school graduates who are prepared 
to fill the hundreds of thousands of unfilled STEM-related jobs in the U.S.8 


In the remainder of this report, we describe activities and progress made this year towards achieving our 
network aim. First, we present our formal network aim and the improvement science tools and methods 
that guide our work. Next, we describe the instructional routines that teachers tested throughout the 
2017-18 school year in support of the aim, before describing the progress made towards achieving our 
aim. Then, we present two sets of lessons learned from network activities. The first set relates to lessons 
about student-centered math instruction, and the second set relates to lessons about how improvement 
knowledge is spread in the context of a network. We conclude with a summary of next steps for the 
2018-19 school year, our third full year as a network. 


Network Aim and Driver Diagram BMTN Aim Statement 


NICs are Organized around a common aim, which the network By 2019, the number 
co-constructs during its initial phase of development. By 
2019, the BMTN aims to increase the number of New England ; i 
students who connect, justify, and solve with depth by 2,019. connect, justify, and 
We selected this number of students when we launched solve with depth will 
our three-year network based on the projected number of increase by 2,019. 
teachers and students who would ultimately participate.° 


foymcinblel=sanncwrtd ate) 


Our efforts to meet this aim by the spring of 2019 are based 
on a common, working theory of improvement, called a 
driver diagram. A driver diagram is a visual tool that NICs use to illustrate the network's working theory 
of improvement as it relates to the network's aim."° It also provides a common language to frame the 
improvement work. Given the network's focus on teachers, we identified math instruction as our primary 
driver (see Exhibit 1). We hypothesize that when teachers provide ongoing opportunities for students to 
connect, justify, and solve with depth, we will make progress towards our aim. Two key dimensions of this 
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work are the tasks teachers use with students and the extent to which students assume responsibility 
for academic content. Because we anticipate the tasks and strategies for shifting academic responsibility 
to students to differ for activities focused on introducing students to new material and activities focused 
on practicing previously introduced material, we focused our secondary drivers on those aspects of 
instruction. Teachers develop change ideas—in our case, specific instructional routines—that they 
hypothesize will increase the depth of student engagement and are aligned to the primary driver and at 
least one secondary driver. 


Exhibit 1. BMTN Driver Diagram 


AIM Statement 


Deep Student 
Engagement in Algebra 


2,019 in 2019 
By 2019, the number of 
students who connect, 
justify, and solve with depth 
in algebra will increase by 
POS) 


Connect. Make connections 
among mathematical 
F) FeXol awa] aalsymee) ale(-) 0lecvar- ale) 
applications to real-world 
contexts, where appropriate. 


Justify. Communicate and 
justify mathematical thinking 
as well as critique the 
reasoning of others. 


Solve. Make sense of and 
solve challenging math 
problems that extend 

beyond rote application of 

algorithm. 


Primary Driver 


(WHAT?) 


Mathematical Instruction 


ules 


connect, justify, olve 


g choice 
of task/activity and by shifting 
the academic responsibility 
to the students. 


(Instruction | 


student-centered 


Secondary Drivers 
(WHERE?) 


Instructional 
Routines to 
Introduce New 
Material 


Takia ge(oid(olar-)| 
Routines to 
Practice/Reinforce 
Previously 
Introduced Material 


Change Ideas 
(HOW?) 


After teachers select change ideas they think will lead to improved student engagement, they subject 
these ideas to rapid-cycle testing in their classrooms. The testing follows the Plan-Do-Study-Act (PDSA) 
improvement science process, which is framed by three questions teachers ask themselves each time 
they test out an idea: (1) Will | implement the instructional routine as planned? (2) Will students engage in 
the activities associated with the instructional routine? and (3) Will students engage with depth? During 
the Plan phase, teachers create a specific plan for how they will implement the routine and collect data 
on the three framing questions. They also make predictions for what they might see in the data. In the 
Do phase, they execute the planned routine and collect associated data. They also note any immediate 
reactions to the activities as implemented. During the Study phase, teachers carefully analyze the 

data, compare it to their predictions, and determine what to do next. In the Act phase, teachers decide 
whether to adopt the instructional routine as is, adapt it in some way, or abandon it altogether. 


Better Math Teaching Network: Lessons Learned from Year2 | 6 


Network Learning Structures 


When teachers join the BMTN, they participate in a weeklong summer workshop where they learn the 
basics of improvement science and PDSA testing, get oriented to the network aim and vision of student- 
centered instruction, and plan for the upcoming school year. Returning BMTN members join the new 
members partway through the summer workshop to share their experiences, discuss instruction, and 
collaboratively plan for PDSA testing. When the school year begins, new and returning teachers begin to 
test their change ideas with PDSA cycles. They typically complete 3-5 PDSA trials over a 4-6 week testing 
period. After each testing period, they meet virtually with a small group of teachers who are working on 

a similar change idea within connect, justify, or solve. During these meetings teachers discuss how similar 
routines played out in different contexts, share what they learned from the data they collected, and get 
feedback from others on the Act phase. These small-group meetings are typically followed by a whole- 
network, in-person meeting. The whole-network meetings provide a larger forum for teachers to share 
what they are learning outside their smaller groups and for the hub to provide professional development 
support on emerging issues as needed. This process of individual testing followed by small-group and 
whole-network meetings repeats three times over the course of the school year. At the end of the school 
year, teachers present their refined instructional routines at a celebratory mini-conference. These refined 
routines are then written up and shared with the next cohort of BMTN teachers during the following 
summer workshop. Exhibit 2 summarizes these activities over a typical school year. 


Exhibit 2. PDSA Testing Process and Timeframe 


=> Es 19.9, 


End of year 
celebration 
teachers present 
refined routines 


ae 


Five in-person meetings per year, anchored by a weeklong summer institute 


Instructional Routines Tested in 2017-18 


During the 2017-18 school year, BMTN teachers tested and refined a variety of instructional routines 
designed to deepen student engagement with algebra content. The routines focused on connect, justify, 
or solve and at least one of the network's secondary drivers: the introduction of new material or the 
reinforcement of previously introduced material. Exhibit 3 lists the six instructional routines BMTN 
teachers tested and refined that focused on deepening students’ mathematical connections. Three of 
the routines were designed for the introduction of new material, one was designed for reinforcement 
of previously learned material, and two were designed to work in either context. 
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Exhibit 3. Instructional Routines Focused on Connect, 2017-18 BMTN Teachers 


Secondary Driver 


TakcidaUletd(e)ar-)mcolenulal=pm@ealatqiee 


Taha) Reinf. 
Reminding students of the big ideas of the unit to support connections Y 
Using a template and class discussion to emphasize connections Y 


Using tasks and guiding questions to support students in making connections 
through the introduction of new material 


Eliciting connections through probing questions Y Y 
Exit routines to build mathematical connections 
Introducing new material by making connections with students’ previous knowledge v 


DATA SOURCE: BMTN teacher change idea summaries, 2017-18 school year. 


Our teachers most commonly focused their improvement efforts on supporting students in justifying 
their thinking and critiquing the reasoning of other students with greater depth. As illustrated in 

Exhibit 4, teachers tested and refined 21 different routines in this area, with the majority aligned to the 
reinforcement of previously introduced material (16 routines) and the remaining 5 designed for use with 
the introduction of new material. Unlike the routines focused on connect, none of the routines focused 
on justify targeted both secondary drivers. 
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Exhibit 4. Instructional Routines Focused on Justify, 2017-18 BMTN Teachers 


Secondary Driver 


Routine 
Talane) Reinf. 


Using claim-evidence-reasoning to support strong justifications Vv 


Using open-ended, scaffolded tasks to help students develop and justify strong 
claims 


Using a structured talk protocol to support justifications 
Infusing opportunities for justification in math tasks 


Using comparison tasks to improve justifications 


—s \ Bae S 


Using non-rote problems to improve student justifications 


Using tasks and a structured whole-class discussion protocol to support justifica- 
tions 


Asking students to order statements to make strong justifications 

Using a small-group protocol to promote deep justifications 

Communicating deep learning from inquiry-based activities Vv 
Using “Which One Doesn't Belong” activities to support deep justification 

Using a protocol and poster template to promote justifications 

Using a problem-solving protocol to support deep justifications 

Using strategize, demonstrate, and explain to support deep justifications 


Critiquing worked examples to improve justification skills 


—— S\ Bee S Bae < 


Using small-group work to improve verbal justifications 

Introducing new material with examples to support justification v 
Sentence frames to support justification Vv 
Using a justification worksheet to support justification v 

A partner share protocol to promote deep justifications 


Tiered checkpoints to promote justifications 


DATA SOURCE: BMTN teacher change idea summaries, 2017-18 school year. 
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Finally, as presented in Exhibit 5, BMTN teachers tested and refined nine instructional routines during the 
2017-18 year that focused on solving with depth. Only one of the solve routines targeted the introduction 
of new material exclusively, while four routines each targeted the reinforcement of previously introduced 
material or both secondary drivers. 


Exhibit 5. Instructional Routines Focused on Solve, 2017-18 BMTN Teachers 


Secondary Driver 


Taksiu a ceid(e)at-]i.ce)0ha) al=Mmcve) hVi-) 
Talaae) Reinf. 


Protocol to support small-group problem solving 

Guiding questions to support problem solving 

Assigning roles and a protocol to support small-group problem solving 
Using challenging homework problems to promote skill in problem solving 


Small-group problem solving protocol to promote depth v 


—— <\ ie SN ie 


Scaffolding and solving cognitively demanding tasks to encourage problem solving 


Providing prompts for adults providing student support in math class to encour- 
age problem solving 


Using protocols to support students in making sense of non-routine problems & 
explaining approach 


Daily mindfulness time in math class to promote problem solving v 


DATA SOURCE: BMTN teacher change idea summaries, 2017-18 school year. 
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Assessing Progress Towards the Aim 


As illustrated in BMTN's driver diagram, the ongoing testing, refinement, and sharing of instructional 
routines is meant to stimulate deep student engagement in algebra. By 2019, which is the third year of 
our network, we aim to increase the number of students who deeply engage in connect, justify, and solve 
by 2,019. Our primary method for tracking progress towards the aim is a student survey, in which we 
collect information about how often students are deeply engaged with algebra content. In addition, to 
assess progress on providing opportunities for deep engagement through math instruction (our primary 
driver), we administer a teacher survey to the BMTN teachers. In this section, we present findings from 
both surveys. 


Network Making Progress Towards Aim 


The student survey consists of 15 items which ask students how often they are engaged in activities 
that are aligned with connect, justify, or solve. Five items address activities associated with connect, five 
items address activities associated with justify, and five items address activities associated with solve. 
In reporting engagement in the activities, students choose from Never, Rarely, Sometimes, Often, and 
Almost Daily. Exhibit 6 lists the survey items along with the reliabilities (Cronbach's Alpha) for these 
items by domain. 


Better Math Teaching Network: Lessons Learned from Year2 | i1 


Exhibit 6. BMTN Student Survey Items, Constructs, and Reliabilities" 


(el geyaley-\el aks 
oq 


Survey Items by Construct and Overall 


Connect. How often... .80 
Do you make sense of mathematical rules, concepts, and relationships? 


Do you make connections to math concepts from other classes you've taken before or in 
the future? 


Do you make connections between math and real-world situations? 

Do you examine why the steps to solving a math problem or following a procedure work? 

Do you make connections to math concepts you learned previously in this class? 

Justify. How often... 82 
Do you explain your answers to others in the class? 

Do you argue or defend your approach to solving math problems? 

Do you critique the mathematical reasoning of others—either written or spoken? 

Do you evaluate other students’ approaches to solving math problems? 

Do you discuss possible solutions to math problems with other students? 

Solve. How often... 78 
Do you keep trying different ways to solve math problems even when they are hard? 

Do you re-read or go over a math problem again if you have trouble understanding it? 

Do you keep working on math problems even when you are stuck? 

Do you determine if your answers to complex math problems make sense? 

Do you solve math problems with multiple steps that take more than 20 minutes to solve? 


Overall 91 


DATA SOURCE: BMTN Student Survey, 2017-18 school year. 
NOTE: We dropped the last item in the Solve domain due to low item-test correlation and alpha—both for this domain and 
overall. 


To compute reliabilities, we assigned 1 point to Never, 2 points to Rarely, 3 points to Sometimes, 4 points to 
Often, and 5 points to Almost Daily. As illustrated in Exhibit 6, the reliabilities for each domain and overall 
are high. Only one of the 15 survey items—“Do you solve math problems with multiple steps that take 
more than 20 minutes to solve?”—was dropped from these analyses because it was poorly correlated 
with the other survey items. 


To assess progress on the aim by domain and overall, we established criteria for “deep engagement” from 
the activities specified in the student survey. Survey response options Never and Rarely clearly indicate 


Better Math Teaching Network: Lessons Learned from Year2 | 12 


limited opportunities for deep student engagement, and Often and Almost Daily indicate more extensive 
opportunities for deep engagement. However, Sometimes could also be an appropriate expectation for 
several of the activities, even in highly student-centered classrooms. For example, we do not necessarily 
expect students to “make connections to math concepts from other classes or in the future” A/most Daily 
or Often. The nature of the topic may not lend itself to daily discussion of connections to other classes or 
future work. We do, however, think that students should be making those types of connections at least 
some of the time. In addition, if a teacher focuses her improvement work on justify, we might expect 
student responses to those items to be higher at the end of the year than for the items associated with 
connect or solve. 


Given these considerations, we determined that responses that average 4 or higher within a domain or 
higher on the survey overall provide strong evidence of deep student engagement and responses that 
average a 3 or provide moderate evidence of deep engagement. 


Exhibit 7 shows the progress that the network made towards meeting its aim during the 2017-18 school 
year. At the end of the school year, about 84 percent of students reported an average of 3 or higher 
(moderate evidence of deep engagement) across the items on the survey. This represents 821 students 
across BMTN classrooms. Among this group, 283 students reported an average of 4 or higher, providing 
strong evidence of deep engagement in algebra. More students reported deep engagement with solve 
than for connect or justify. A total of 91 percent of BMTN students reported an average of 3 or more and 
48 percent reported an average of 4 or more. 


Exhibit 7. Percentage and Number of Students Deeply Engaged in Algebra, by Evidence Level 
and Dimension, 2017-18 School Year 


91% 
* me 
zzz SSS (Saas 
Overall sa students 
EN sesionts| | | 
a ae Students 


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 


|| Moderate Evidence of Deep Engagement 


|| Strong Evidence of Deep Engagement Data Source: BMTN Student Survey, spring 18; N = 977. 


DATA SOURCE: BMTN Student Survey, spring 18; N = 977. 


When we look at student engagement across the two years of the network (2016-17 and 2017-18), 
we can see our progress in reaching the aim. Exhibit 8 shows the number of students who reported 
an average of 3 or higher and 4 or higher on the student survey overall in 2016-17, 2017-18, and in 
total. Across the two years, a total of 1,197 students provided at least moderate evidence of deep 
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engagement in algebra. This is 59 percent of the 2,019 students we aim to deeply engage with algebra by 
the year 2019. A total of 413 students (21 percent of 2,019 students) provided strong evidence of deep 
engagement. As the network moves into the 2018-19 school year, with additional teachers and students 
in the current cohort, the BMTN is well positioned to meet its aim by 2019 at the moderate level of 
evidence and increase the number of students at the strong level. 


Exhibit 8. Number of Students Deeply Engaged in Algebra, by Evidence Level 
and Network Year 


1000 1200 


Net Moderate Evidence of Deep Engagement a Strong Evidence of Deep Engagement 


DATA SOURCE. BMTN Student Survey, spring 2017 (N = 977) and spring 2018 (N = 447). 


Teachers Increased Opportunities for Deep 
Student Engagement 


As a process measure of the extent to which our teachers are providing opportunities for deep 
engagement, we also surveyed teachers about the frequency that they provided students with the 
specific opportunities students reported on the student survey. For example, where the student survey 
asks, “How often do you make sense of mathematical rules, concepts, and relationships?” the teacher 
survey asks, “How often do you provide students with opportunities to make sense of mathematical rules, 
concepts, and relationships?” The response options on the teacher survey were the same as for the 
student survey: Never, Rarely, Sometimes, Often, and Almost Daily. Like on the student survey, we consider 
an average score of 3 or more to provide moderate evidence of opportunities for deep engagement and 
a score of 4 or more to provide strong evidence of opportunities for deep engagement. 


All BMTN teachers indicated that they offered the opportunities for student engagement in each of the 

15 items at least Sometimes. This is encouraging but perhaps expected given the active involvement of 
teachers in PDSA testing during the 2017-18 school year. Also, as illustrated in Exhibit 9, the percentage 
of teachers who reported strong evidence of providing opportunities for deep engagement varied by the 
time of year of the survey (Fall or Spring) and the domain of engagement (connect, justify, or solve). By 
domain and overall, the percentage of teachers who offered students deep opportunities for engagement 
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at the strong evidence level increased from fall to spring, with almost half of BMTN teachers (49 percent) 
providing such opportunities in spring 2018. Among the domains, like the student survey, solve had the 
strongest evidence, with almost 70 percent of BMTN teachers reaching the strong level by Spring 2018. 


Exhibit 9. Percentage of Teachers Providing Opportunities for Deep Student Engagement 
at the Strong Level, Fall to Spring, 2017-18 School Year 


Connect 
Justify 
Solve 
Overall 


0% 10% 20% 30% 40% 50% 60% 70% 80% 


Spring 208 Fall 2017 


DATA SOURCE: BMTN Teacher Survey, 2017-18 school year, N = 35. 
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Lessons Learned 


As described in our report about the network's first year of implementation, we completed the chartering 
phase of NIC development at the end of the 2015-16 school year and moved into the network learning 
phase in 2016-17. In the chartering phase, we convened a small group of researchers and practitioners 
to define the problem and determine the way in which we would address it through a NIC.'* During the 
network learning phase, we used PDSA cycles to test and refine student-centered instructional routines 
and began to consolidate our learning, which we continued into 2017-18 and described earlier in the 
report. As we moved further into this phase in 2017-18, we also began to move into the final phase of NIC 
development: spreading knowledge gained through PDSA testing with other teachers inside and outside 
of the network. In this section, we present what we learned this year about student-centered instruction 
and how to support the spread and continued testing of instructional routines that support deep student 
engagement with algebra. 


Lessons Learned Related to Student-Centered Instruction 


This year’s testing of instructional routines drew upon what we learned about student-centered 
instruction the previous year, namely that instruction should include tasks and scaffolds that provide 
opportunities for deep engagement in making connections, justifications, and solving non-rote problems. 
As new and returning teachers tested and refined instructional routines during the 2017-18 school year, 
they used tasks and scaffolds to support deep student engagement. Through that testing, teachers 
learned more about the nature and implementation of the tasks and scaffolds that are needed to 
improve the depth of students’ connections, justifications and problem-solving processes. Specifically, 
our teachers learned that they needed to: 


¢ Focus instructional tasks and activities on mathematical relationships, as opposed 
to rote procedures; 


e Use questions that explicitly address math relationships to assess the depth of 
student connections, justifications, or problem-solving processes; 
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e Allow time for students to work individually to gather their thoughts before working 
with other students; 


e Provide opportunities to see examples of, practice, and receive feedback on their 
attempts at deep engagement; and 


e Make sure they had a deep understanding of the content themselves to design 
and implement instruction with these features. 


Whether working on connect, justify, or solve, teachers found that the more the tasks and other 
instructional activities focused on mathematical relationships, the greater the opportunity for students 
to demonstrate deep engagement. If, for example, the tasks or instructional activities focused on 
memorizing and applying algebraic procedures, such as solving an equation, there was little opportunity 
for students to make a deep mathematical connection, provide a deep justification, or engage in deep 
problem solving. If, however, the task or instructional activities emphasized the relationship between 
the solution to an equation and a real-world context, a graph, or a table, students had more opportunity 
to identify a deep connection, create a deep justification, or engage in deep problem solving. Exhibit 

10 shows examples of the tasks our teachers used to focus on math relationships and promote deep 
engagement with the content. 


Exhibit 10. Tasks that Focus on Math Relationships for Connect, Justify, and Solve 


Task to promote deep 
connections 


(Developed by a BMTN 
teacher) 


Task to promote deep 
justifications 


(Edited from Better 
Lesson’) 


Task to promote deep 
problem solving 


(Edited from 
Illustrative 
Mathematics) 


In the “Saving for a Bike” task Jenny started with $50 in her savings account 
and saved $5 per week. You and your classmates developed the following 
rule to represent how much she saved: 
y=5x+ 50 
a. Howis the equation y = 5x + 50 related to the linear function 
f(x) = 5x + 50? 
b. When asked how much money Jenny would save after 8 weeks, 
you and your classmates decided to do the following: 
y=5(8) +50 
« How is this equation related to what you know about inputs and 
outputs for functions? 
c. When asked how long it would take for Jenny to save $185, you 
and your classmates decided to do the following: 
185 = 5x + 50 
« How is this equation related to what you know about inputs and outputs? 


John and his father ran a 100 meter race. John started the race 3 seconds 
after his father. The graph below shows how far the two ran over time. 


90 John 
80 Father 


5 10 15 20 


Answer the following questions as completely as you can: 
1. Who won the race? 
won the race. 


2. I know this because 


The table below shows two coordinate pairs (x, y) that satisfy the equation 
y = mx + b for some numbers m and b. 


I. 
If m = 7, determine possible values for yl and y2. Explain your choices. 


II. 

Find another pair of y-values that could work for m = 7. Explain why they 
would work. How do these y-values compare to the first pair you found for 
m=7? 


DATA SOURCE: BMTN change idea summaries, 2017-18 school year. 
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When they wanted to assess the extent to which students were deeply engaging with the content as 
they worked with math relationships, our teachers learned they needed to use very specific questions. 
For example, teachers learned that asking, “What connections did you make?” or “How do you solve 
that?” does not necessarily elicit deep responses from students, even if the students are deeply engaged. 
Instead, BMTN teachers learned that they need to explicitly address the mathematical ideas in their 
questions to assess the depth of student engagement. Exhibit 11 shows sample questions they used 

for this purpose. 


Exhibit 11. Questions to Elicit and Assess Deep Engagement with Algebra Content 


What connections do you see between the shapes puzzles and systems of equations? 
How does recognizing the pattern help you determine the value of the negative exponents? 
What is the new idea we discussed today? How is it connected to what you already know? 


Student A got one answer and student B got another answer. Who is correct? Explain 
and support your answer with evidence. 


How do you know your answer is correct? 


What information is important to solving the task? Why? 


What solution path will you take to solve the problem? Why? 


DATA SOURCE: BMTN change idea summaries, 2017-18 school year. 


To maximize student engagement with the tasks and explicit questions they were using to support deep 
engagement, our teachers found that they needed to provide time for students to work individually 
before working with other students. This “individual think time” offered the opportunity for a// students 
to engage with the content before they moved to small- or whole-group discussion and heard how 
others were thinking. Teachers also found that the amount time needed for individual processing varied 
depending on the type of instructional routine and complexity of the task or question. 


Even when they implemented instructional tasks that focused on math relationships, used questions 
that targeted specific math content to assess deep engagement, and provided students with “individual 
think time,” our teachers found that they need to do more to support students in understanding what 
constitutes deep engagement. For students to demonstrate deep engagement, they needed to know 
what deep engagement looks like. Specifically, students needed to see examples of deep connections, 
deep justifications, and deep problem-solving processes and they needed opportunities to practice 
demonstrating deep engagement with feedback. Exhibit 12 shows some of the approaches teachers 
used to provide these opportunities. 
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Exhibit 12. Opportunities to Understand and Build Skill in Demonstrating Deep Engagement 


Strategy Description 


Provide samples of connections, justifications and of problem solving processes. 
Critique Examples Ask students to critique the depth of the work. 

As a group identify characteristics and/or introduce a rubric to evaluate deep 

engagement. 


Provide a series of sentence frames to help students demonstrate deep 


ntence Fram Peas, : 
SSMEUIE SATIS engagement. For example: “I think the answer is because 


” 


Provide a graphic organizer to support writing. For example: 


CLAIM (your answer to the question prompt) 


Subclaim Evidence Reasoning/Justification 
(reasons your claim is (comes from the math (explaining your 
true) you did and math evidence and 
concepts) connecting it back to 
Graphic Organizers your claim) 


Possible sentence starters: Possible sentence starters: | Possible sentence starters: 


+ “One reason that + “When looking - “This shows that...” 
supports my claim at___” + “This proves that...” 
iss - “The evidence to 

+ “This is true support this comes 
because___” from__” 


Provide a series of sentences and ask students to order the statements so that 


Ordering Statements 
they clearly demonstrate deep engagement. 


Collect student responses. 
Practice with Feedback Use a rubric to provide feedback to improve the depth of student responses. 
Students use the feedback to create new responses. 


DATA SOURCE: BMTN teacher change idea summaries, 2017-18 school year. 
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Finally, our teachers learned that to design and implement instruction that promotes deep student 
engagement, they needed to have a deep understanding of the math content for themselves. For 
example, to pick a task that is ripe for making deep connections, providing a deep justification, 

or engaging in deep problem solving, teachers needed to know what math relationships the task 
should target and how one might explain those relationships. To support teachers in deepening 

their understanding of math relationships, we organized a collection of book study groups that read 
and discussed one of two books from the National Council of Teachers of Mathematics’ Essential 
Understandings series.'* Each book in the series focuses on the “big ideas” and related “essential 
understandings” that are central to a math topic recognized as challenging to teach and learn. A majority 
of BMTN teachers (24 of the 41) elected to join a study group. Each teacher in the group took turns 
leading the group. We did not attend the sessions, but we did ask teachers to record and document 
activity within those session, Participating teachers reported high levels of satisfaction with the content 
and structure. Exhibit 13 provides an overview of the book study group activity. 


Exhibit 13. BMTN Book Study Groups 


Impetus: As teachers delved further into their PDSA testing, they decided that to promote student 
understanding of mathematical relationships, they needed to take some time to solidify their own 
understanding of some of the major concepts in algebra. They showed interest in forming small book 
study groups after a whole network meeting activity in December 2017. 


Participants: A total of 24 of the 41 BMTN teachers participated in the voluntary book study groups. 
Participants included new and returning network teachers, and each group had three or four members. 


Topics and Structure: The study groups were formed based on their availability to meet and their interest 
in one of two books: (1) Functions, which was targeted to students in grades 9-12, and (2) Expressions, 
Equations, and Functions, which was targeted to students in grades 6-8. 


We provided each group with a suggested structure, though groups had leeway in how much they decided 
to read and how often they met. Groups met virtually 5-7 times, with each meeting lasting about one hour. 
We encouraged groups to use the following discussion questions for each meeting: 


1. What are your initial reactions? Did you learn anything new—or think about something 
familiar differently—based on what you read? If so, what was new? 


2. Did you have any questions related to the Reflect activities you completed? If so, 
what were they? 


3. To what extent did the content you read about align with how this content is presented 
in your curriculum/textbook? 


4. Are there any lingering questions? 


We also initially assigned each group a meeting facilitator and a note taker. The facilitator made sure that 
the discussion questions were posed and that each person had the opportunity to respond. The note taker 
captured a high-level overview of what was discussed and documented each meeting in a shared group 
folder. We encouraged groups to take turns playing each of these roles. 


Teacher Perceptions: Based on informal interviews and a review of archival information from the meetings 
(e.g., meeting notes, recordings of the virtual meetings), teachers actively participated in the study groups 
and found the meetings valuable. We plan to offer study groups again during the 2018-19 school year. 
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Lessons Learned Related to Network Structures 
to Support Spread 


As we increased the size of our network and moved into the second year of implementation (2017-18), 
we began to focus on spreading (a) the refined student-centered instructional routines that were 
developed through PDSA testing in the first year and (b) structures for supporting continued testing of 
those instructional routines inside and outside of the network. Specifically, we shared the refined routines 
with new and returning members in the network, encouraged teachers to test those routines or a version 
of them, and put structures in place to support new members in learning about the PDSA process. In 
addition, we shared the refined routines with instructional leaders outside of the network and worked 
closely with one leader to facilitate a professional learning community (PLC) of teachers in her district. 
The PLC focused on testing one of those routines and learning about improvement science. Through this 
work, we learned the following: 


* To promote implementation and continued refinement of instructional routines in new 
contexts, provide the “refined” routine as well as a list of “key learnings” about how to 
support deep student engagement that emerged during the testing process; 


¢ To support spread within the network, provide summaries of the routines that were 
tested and refined the previous year and place new and returning members in the same 
testing groups; and 


¢ To support spread outside of the network, provide a routine to test and guide teachers 
through the PDSA process with that routine. 


At the end of the first year of the network (2016-17), each BMTN teacher created a “change idea 
summary,” which synthesized findings from a full year of PDSA testing. The change idea summaries gave 
an overview of the problem they were trying to solve, described the change idea they tested, provided 

a detailed outline of the “refined” routine they recommended after repeated testing, included the data 
they collected that showed evidence of promise, and offered advice for implementing the refined routine. 
These change idea summaries were intended to be a mechanism for spreading the refined routine to 
other teachers inside and outside of the network. 


As we shared the change idea summaries with others, we learned that teachers were not necessarily 
ready to implement the refined routine as described in the change idea summary. Teachers teach in 
different contexts and use different instructional approaches. Some teachers work with students who 
have experienced student-centered instruction throughout their educational careers. Others work 

with students who are used to lecture-style instruction. Some teachers already use student-centered 
instructional approaches in their classrooms. Others are just starting to use those approaches. Each 
refined routine was the result of a full year of testing in a specific context by a teacher who had a specific 
instructional approach. In addition to teaching in a potentially different context with a potentially different 
instructional approach, teachers trying the routine for the first time have not spent a full year refining 
the routine to match their students. This makes it difficult for teachers to implement the refined routine 
as described. 


Given these challenges, we found it helpful to provide teachers with a list of “key learnings” as well as the 
change idea summary that included the refined routine. These key learnings highlight important findings 
that emerged and contributed to the refinement of the routine over the year of testing. For example, as 

described in the previous section, teachers learned that it is important to provide individual think time on 
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a new task before moving to group work. This could be listed as a key learning on teachers’ change idea 
summaries. If teachers are not ready to implement the refined routine as described, they can implement 
a modified version that attends to the key learnings but that can be more reasonably implemented in 
their context, given their background. Exhibit 14 shows an example. 


Exhibit 14. Excerpt from a Change Idea Summary that Includes Key Learnings 


Problem: 

Justification does not come naturally to students and they are rarely asked to do so before coming into 
our classrooms. | have found that students often want to get to the solution, but do not necessarily work 
to understand beyond the procedure to solve or the reasonableness of the solution. 


Change Idea: 

| want to improve my students’ ability to justify their own learning and understanding of material by 
presenting problems that will allow student disagreement or a need to prove that the result is rational. 

| am going to incorporate more unfamiliar problems that have multiple entry points into my instruction 
and provide opportunities for students to communicate and justify their thought process and approach. 


Key Learnings: 
*  Itis important to give students individual thinks time before they work in small groups. 
* Students worked best either with a partner or a group of three. 
* Students need to know the expectations and what a good justification is to be able to give 
meaningful justification. 
+ Not all tasks have to have real world applications to qualify as a justification task. 


Final Routine: 

1. Give the students a problem or task with multiple entry points that incorporate the 
students’ new and old math knowledge. 

2. Have the students read through the problem and give them private reasoning time before 
moving to group work. 

3. (If the students are struggling | ask them to write down questions that they have.) 
Give anywhere between 5 to 10 minutes. 

4. | look at how engaged the class is with the task to determine the time. 

5. Allow students to work in small groups (arrange them) or with their table partner. 
Give students one minute each to share their thoughts or questions on the topic. 
Give the group two more minutes to see if they can answer or address their new learning. 

6. If students still have questions they may write them on a post-it and put them in the 
parking lot of questions for the class or teacher to answer. (Not all questions in the parking 
lot have to be answered, but some may need to be answered for students to continue 
with their work.) 

7. Give students 5 to 10 minutes to continue the tasks if they haven't finished. Make sure you 
give time to students to justify their solution. 


DATA SOURCE: BMTN change idea summaries, 2017-18 school year. 


Within the network, we supported new members in implementing modified versions of the refined 
routines or new routines altogether by providing opportunities to work with returning members in 

PDSA testing groups. We learned that by including both new and returning members in the same testing 
groups, new members learned about the PDSA process—including how to measure improvement—and 
gathered ideas for providing students opportunities to deeply engage with content. As hub leaders, we 
still attended the PDSA testing meetings and served as coaches. As we facilitated the meetings, we found 
that the returning members’ contributions enhanced the learning of the new members. 
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Outside the network, we used the key learnings to modify two refined routines that could most easily 

be implemented in a high school math classroom regardless of context or instructional approach and 
shared them with the ten instructional leaders who were affiliated with the network. We encouraged 

the instructional leaders to share the routines with other teachers and provide feedback regarding how 
things went when they implemented the routines. One instructional leader shared the routine with a few 
teachers and expressed interest in forming a PLC in her district focused on implementing the routine 
and using PDSA cycles to improve it. We partnered with her and organized a series of PLC sessions, with 
each session addressing a step in the PDSA cycle, using the modified routine. In other words, teachers 
learned about PDSA testing and completed the Plan step in the first session, tested and collected data on 
the routine in their classrooms, completed the Do and Study steps in the next session, continued testing 
in their classroom, and discussed next steps in the final session. This approach of providing a modified 
routine that was based on the refined routine and associated key learnings and using that routine to 
learn about the steps of the PDSA process worked well for spreading the routine and the PDSA process 
to others. Exhibit 15 provides more information on the PLC work. 


Exhibit 15. PLC Work with Teachers Outside of the Network 


Impetus: We were interested in learning how to spread refined routines and the PDSA process to teachers 
outside of the network. One instructional leader was interesting in working with middle and high school 
teachers to implement and continue to improve instructional routines using PDSA testing. 


Participants: One instructional leader, who is the department chair at a high school in New England; four 
math teachers at that high school; six math teachers from the feeder middle school. 


Topics and Structure: We organized a series of three PLC sessions, each led by a BMTN hub member, 
with work periods in between each session. During each work period, teachers tested a routine that was 
developed by one of the teachers in the BMTN. 


PLC Session #1 ¢ Learn about the BMTN, including the problem we are solving, 
our aim, and the driver diagram 


¢ Learn how to use PDSA testing to make improvements 


¢ Learn about a refined instructional routine that was developed 
by one of the BMTN members 


Work Period #1 Implement the instructional routine and collect data (Plan, Do) 


PLC Session #2 ¢ Analyze the data using a BMTN rubric and use the analysis to 
decide what modifications to make (Study, Act) 


e Learn about a second refined routine that was developed by 
a BMTN member 


Work Period #2 Test and refine one of the two refined routines using the 


PDSA process 


PLC Session #3 Share learning from the testing and consider next steps 


Teacher Perceptions: Teachers enjoyed having the opportunity to discuss instruction. They felt the PDSA 
process provided them an opportunity to look more closely at instruction and student engagement. 
Specifically, they learned to look beyond a student's answer to determine whether that student had deeply 
engaged with the content. All teachers were interested in continuing the next school year. 
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Where We Are Headed Next 


As we reflect on the lessons learned from our network's second year, we have identified areas for continued 
growth and for new growth. We plan to continue to leverage the experience and expertise of our returners and 
the support structures for small-group and whole-network learning. We plan to reach new teachers through PLC 
meetings similar to the model we piloted in 2017-18 and described in Exhibit 15. As we expand, we anticipate 
learning more about what student-centered instruction looks like, how to enact it, and how to spread promising 
instructional routines. We also anticipate that our growth will provide additional opportunities for teacher 
leadership. 


More specifically, we view teacher leadership as critical to the success of the network, but also important to 

the profession overall. Our returning teachers will assume several types of leadership roles in the coming school 
year. Several returners will facilitate small-group PDSA testing meetings, after receiving an initial training with 
ongoing support from the hub. Others will be leading PLC meetings at their schools with other math teachers, 
following the model we piloted this year. Others plan to make formal presentations to school and district leaders 
about how improvement science and instruction-oriented improvement networks function. And we expect more 
of our returners to continue to present their BMTN work at state, regional, and national conferences. 


We expect to continue to learn how the spread of knowledge best occurs inside and outside the network. This 
report describes some initial learnings about spread, but as we reach more teachers, we expect to learn a great 
deal more. This knowledge will be critical to supporting our goal of deeply engaging an increasing number of 
students in algebra. 


Finally, as we share our work within the broader (and growing) community of improvement networks, we are 
mindful that a limited number of networks are focused primarily on improving instruction. As we work closely 
with teachers on testing and refining routines within complex classroom environments, we hope to contribute 
useful information about such networks in the improvement science community. We also think that our work 
might support other strands of work that need further research and development, including student-centered 
instruction, teacher professional development, teacher leadership, and research-practice partnerships. 
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A report describing our first year of implementation can be found here: https://www.nmefoundation.org/ 
resources/assessment/the-better-math-teaching-network-lessons-learned. 
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